<!--
Copyright (c) 2019-2026, Hossein Moein
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
  notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
  notice, this list of conditions and the following disclaimer in the
  documentation and/or other materials provided with the distribution.
* Neither the name of Hossein Moein and/or the DataFrame nor the
  names of its contributors may be used to endorse or promote products
  derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL Hossein Moein BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-->
<!DOCTYPE html>
<html>

<head>
<style>
body {
  background-image: linear-gradient(Azure, AliceBlue, GhostWhite, WhiteSmoke);
}
</style>
</head>

<body style="font-family: Georgia, serif">
  <font size="+3">&#8592;</font> <a href="https://hosseinmoein.github.io/DataFrame/docs/HTML/DataFrame.html">Back to Documentations</a><BR><BR>
  
  <table border="1">

    <tr bgcolor="lightblue">
      <th>Signature</th> <th>Description</th> <th>Parameters</th>
    </tr>
    <tr bgcolor="Azure">
      <td>
<pre class="code_syntax" style="color:#000000;background:#ffffff00;"><span class="line_wrapper"><span style="color:#004a43; ">#</span><span style="color:#004a43; ">include </span><span style="color:#800000; ">&lt;</span><span style="color:#40015a; ">DataFrame/DataFrameMLVisitors.h</span><span style="color:#800000; ">&gt;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper"><span style="color:#800000; font-weight:bold; ">template</span><span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">typename</span> T<span style="color:#808030; ">,</span> <span style="color:#800000; font-weight:bold; ">typename</span> I <span style="color:#808030; ">=</span> <span style="color:#800000; font-weight:bold; ">unsigned</span> <span style="color:#800000; font-weight:bold; ">long</span><span style="color:#808030; ">,</span></span>
<span class="line_wrapper">         <span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">size_t</span> A <span style="color:#808030; ">=</span> <span style="color:#008c00; ">0</span><span style="color:#800080; ">&gt;</span></span>
<span class="line_wrapper"><span style="color:#800000; font-weight:bold; ">struct</span> AnomalyDetectByLOFVisitor<span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper"><span style="color:#696969; ">// -------------------------------------</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper"><span style="color:#800000; font-weight:bold; ">template</span><span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">typename</span> T<span style="color:#808030; ">,</span> <span style="color:#800000; font-weight:bold; ">typename</span> I <span style="color:#808030; ">=</span> <span style="color:#800000; font-weight:bold; ">unsigned</span> <span style="color:#800000; font-weight:bold; ">long</span><span style="color:#808030; ">,</span></span>
<span class="line_wrapper">         <span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">size_t</span> A <span style="color:#808030; ">=</span> <span style="color:#008c00; ">0</span><span style="color:#800080; ">&gt;</span></span>
<span class="line_wrapper"><span style="color:#800000; font-weight:bold; ">using</span> and_lof_v <span style="color:#808030; ">=</span> AnomalyDetectByLOFVisitor<span style="color:#800080; ">&lt;</span>T<span style="color:#808030; ">,</span> I<span style="color:#808030; ">,</span> A<span style="color:#800080; ">&gt;</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span></pre>
      </td>
      <td>
        This is a "single action visitor", meaning it is passed the whole data vector in one call and you must use the single_act_visit() interface.<BR><BR>

        Local outlier factor (LOF) is an algorithm used for Unsupervised outlier detection. It produces an anomaly score that represents data points which are outliers in the data set. It does this by measuring the local density deviation of a given data point with respect to the data points near it. Local density is determined by estimating distances between data points that are neighbors (k-nearest neighbors). So for each data point, local density can be calculated. By comparing these we can check which data points have similar densities and which have a lesser density than its neighbors. The ones with the lesser densities are considered as the outliers. Firstly, k-distances are distances between points that are calculated for each point to determine their k-nearest neighbors. The 2nd closest point is said to be the 2nd nearest neighbor to the point.<BR>
        LOF is unlike other outlier detection algorithms. It doesn't work on proximity of data, but it works on density. For that reason, it may not work for all datasets (see code sample below) and it may need some experimenting. To do effective LOF application consider the following points:
        <OL>
          <LI>You must know the nature of your data</LI>
          <LI>You must understand the LOF algorithm well</LI>
          <LI>You must fine tune the parameters <I>k</I> and <I>threshold</I></LI>
        </OL>

        <BR>
        The result is a vector of indices to the original data that were deemed outliers.<BR>

    <I>
    <PRE>
using distance_func = std::function&lt;double(const T &amp;x, const T &amp;y)&gt;;

AnomalyDetectByLOFVisitor(
    size_type k,
    value_type threshold,
    <a href="https://hosseinmoein.github.io/DataFrame/docs/HTML/NormalizeVisitor.html">normalization_type</a> norm_type = normalization_type::none,
    distance_func &amp;&amp;f = [](const value_type &x, const value_type &y) -> double  {
                            return (std::fabs(x - y));
                        }));

<B>k</B>: K neighborhood for knn operation
<B>threshold</B>: The threshold for LOF score above which it is considred an anomaly
<B>norm_type</B>: Normalization type. the default is no normalization
<B>f</B>: Function to calculate distance between two datapoints
    </PRE>
    </I>
      </td>
      <td width="30%">
        <B>T</B>: Column data type<BR>
        <B>I</B>: Index type<BR>
        <B>A</B>: Memory alignment boundary for vectors. Default is system default alignment<BR>
      </td>
    </tr>

  </table>

<pre class="code_syntax" style="color:#000000;background:#ffffff00;"><span class="line_wrapper"><span style="color:#800000; font-weight:bold; ">static</span> <span style="color:#800000; font-weight:bold; ">void</span> test_AnomalyDetectByLOFVisitor<span style="color:#808030; ">(</span><span style="color:#808030; ">)</span>  <span style="color:#800080; ">{</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    <span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">cout</span> <span style="color:#808030; ">&lt;</span><span style="color:#808030; ">&lt;</span> <span style="color:#800000; ">"</span><span style="color:#0f69ff; ">\n</span><span style="color:#0000e6; ">Testing AnomalyDetectByLOFVisitor{ } ...</span><span style="color:#800000; ">"</span> <span style="color:#808030; ">&lt;</span><span style="color:#808030; ">&lt;</span> <span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">endl</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    StrDataFrame    ibm<span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    <span style="color:#800000; font-weight:bold; ">try</span>  <span style="color:#800080; ">{</span></span>
<span class="line_wrapper">        ibm<span style="color:#808030; ">.</span><span style="color:#603000; ">read</span><span style="color:#808030; ">(</span><span style="color:#800000; ">"</span><span style="color:#0000e6; ">IBM.csv</span><span style="color:#800000; ">"</span><span style="color:#808030; ">,</span> io_format<span style="color:#800080; ">::</span>csv2<span style="color:#808030; ">)</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper">    <span style="color:#800080; ">}</span></span>
<span class="line_wrapper">    <span style="color:#800000; font-weight:bold; ">catch</span> <span style="color:#808030; ">(</span><span style="color:#800000; font-weight:bold; ">const</span> DataFrameError <span style="color:#808030; ">&amp;</span>ex<span style="color:#808030; ">)</span>  <span style="color:#800080; ">{</span></span>
<span class="line_wrapper">        <span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">cout</span> <span style="color:#808030; ">&lt;</span><span style="color:#808030; ">&lt;</span> ex<span style="color:#808030; ">.</span>what<span style="color:#808030; ">(</span><span style="color:#808030; ">)</span> <span style="color:#808030; ">&lt;</span><span style="color:#808030; ">&lt;</span> <span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">endl</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper">        <span style="color:#800080; ">::</span><span style="color:#603000; ">exit</span><span style="color:#808030; ">(</span><span style="color:#808030; ">-</span><span style="color:#008c00; ">1</span><span style="color:#808030; ">)</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper">    <span style="color:#800080; ">}</span></span>
<span class="line_wrapper">    ibm<span style="color:#808030; ">.</span>get_column<span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">double</span><span style="color:#800080; ">&gt;</span><span style="color:#808030; ">(</span><span style="color:#800000; ">"</span><span style="color:#0000e6; ">IBM_Adj_Close</span><span style="color:#800000; ">"</span><span style="color:#808030; ">)</span><span style="color:#808030; ">[</span><span style="color:#008c00; ">502</span><span style="color:#808030; ">]</span> <span style="color:#808030; ">=</span> <span style="color:#008000; ">800.0</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper">    ibm<span style="color:#808030; ">.</span>get_column<span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">double</span><span style="color:#800080; ">&gt;</span><span style="color:#808030; ">(</span><span style="color:#800000; ">"</span><span style="color:#0000e6; ">IBM_Adj_Close</span><span style="color:#800000; ">"</span><span style="color:#808030; ">)</span><span style="color:#808030; ">[</span><span style="color:#008c00; ">1001</span><span style="color:#808030; ">]</span> <span style="color:#808030; ">=</span> <span style="color:#008000; ">900.0</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper">    ibm<span style="color:#808030; ">.</span>get_column<span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">double</span><span style="color:#800080; ">&gt;</span><span style="color:#808030; ">(</span><span style="color:#800000; ">"</span><span style="color:#0000e6; ">IBM_Adj_Close</span><span style="color:#800000; ">"</span><span style="color:#808030; ">)</span><span style="color:#808030; ">[</span><span style="color:#008c00; ">2002</span><span style="color:#808030; ">]</span> <span style="color:#808030; ">=</span> <span style="color:#008000; ">450.0</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper">    ibm<span style="color:#808030; ">.</span>get_column<span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">double</span><span style="color:#800080; ">&gt;</span><span style="color:#808030; ">(</span><span style="color:#800000; ">"</span><span style="color:#0000e6; ">IBM_Adj_Close</span><span style="color:#800000; ">"</span><span style="color:#808030; ">)</span><span style="color:#808030; ">[</span><span style="color:#008c00; ">5000</span><span style="color:#808030; ">]</span> <span style="color:#808030; ">=</span> <span style="color:#008000; ">1.5</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper">    ibm<span style="color:#808030; ">.</span>get_column<span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">double</span><span style="color:#800080; ">&gt;</span><span style="color:#808030; ">(</span><span style="color:#800000; ">"</span><span style="color:#0000e6; ">IBM_Adj_Close</span><span style="color:#800000; ">"</span><span style="color:#808030; ">)</span><span style="color:#808030; ">[</span><span style="color:#008c00; ">5030</span><span style="color:#808030; ">]</span> <span style="color:#808030; ">=</span> <span style="color:#008000; ">20.0</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    and_lof_v<span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">double</span><span style="color:#808030; ">,</span> <span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">string</span><span style="color:#800080; ">&gt;</span>  anomaly <span style="color:#800080; ">{</span> <span style="color:#008c00; ">10</span><span style="color:#808030; ">,</span> <span style="color:#008000; ">2.0</span> <span style="color:#800080; ">}</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    ibm<span style="color:#808030; ">.</span>single_act_visit<span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">double</span><span style="color:#800080; ">&gt;</span><span style="color:#808030; ">(</span><span style="color:#800000; ">"</span><span style="color:#0000e6; ">IBM_Adj_Close</span><span style="color:#800000; ">"</span><span style="color:#808030; ">,</span> anomaly<span style="color:#808030; ">)</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    <span style="color:#800000; font-weight:bold; ">const</span> <span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">vector</span><span style="color:#800080; ">&lt;</span><span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">size_t</span><span style="color:#800080; ">&gt;</span>  result <span style="color:#808030; ">=</span> <span style="color:#800080; ">{</span> <span style="color:#008c00; ">502</span><span style="color:#808030; ">,</span> <span style="color:#008c00; ">1001</span><span style="color:#808030; ">,</span> <span style="color:#008c00; ">2002</span><span style="color:#808030; ">,</span> <span style="color:#008c00; ">5000</span> <span style="color:#800080; ">}</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    assert<span style="color:#808030; ">(</span><span style="color:#808030; ">(</span>anomaly<span style="color:#808030; ">.</span>get_result<span style="color:#808030; ">(</span><span style="color:#808030; ">)</span> <span style="color:#808030; ">=</span><span style="color:#808030; ">=</span> result<span style="color:#808030; ">)</span><span style="color:#808030; ">)</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    <span style="color:#800000; font-weight:bold; ">constexpr</span> <span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">size_t</span>   item_cnt <span style="color:#808030; ">=</span> <span style="color:#008c00; ">1024</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper">    MyStdDataFrame          df2<span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    df2<span style="color:#808030; ">.</span>load_index<span style="color:#808030; ">(</span>MyStdDataFrame<span style="color:#800080; ">::</span>gen_sequence_index<span style="color:#808030; ">(</span><span style="color:#008c00; ">0</span><span style="color:#808030; ">,</span> item_cnt<span style="color:#808030; ">,</span> <span style="color:#008c00; ">1</span><span style="color:#808030; ">)</span><span style="color:#808030; ">)</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    <span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">vector</span><span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">double</span><span style="color:#800080; ">&gt;</span>   sine_col<span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    sine_col<span style="color:#808030; ">.</span>reserve<span style="color:#808030; ">(</span>item_cnt<span style="color:#808030; ">)</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper">    <span style="color:#800000; font-weight:bold; ">for</span> <span style="color:#808030; ">(</span><span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">size_t</span> i <span style="color:#808030; ">=</span> <span style="color:#008c00; ">0</span><span style="color:#800080; ">;</span> i <span style="color:#808030; ">&lt;</span> item_cnt<span style="color:#800080; ">;</span> <span style="color:#808030; ">+</span><span style="color:#808030; ">+</span>i<span style="color:#808030; ">)</span>  <span style="color:#800080; ">{</span></span>
<span class="line_wrapper">        sine_col<span style="color:#808030; ">.</span>push_back<span style="color:#808030; ">(</span><span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">sin</span><span style="color:#808030; ">(</span><span style="color:#008000; ">2.0</span> <span style="color:#808030; ">*</span> M_PI <span style="color:#808030; ">*</span> i <span style="color:#808030; ">/</span> <span style="color:#008000; ">20.0</span><span style="color:#808030; ">)</span><span style="color:#808030; ">)</span><span style="color:#800080; ">;</span> <span style="color:#696969; ">// Base sine wave</span></span>
<span class="line_wrapper">        <span style="color:#800000; font-weight:bold; ">if</span> <span style="color:#808030; ">(</span>i <span style="color:#808030; ">%</span> <span style="color:#008c00; ">30</span> <span style="color:#808030; ">=</span><span style="color:#808030; ">=</span> <span style="color:#008c00; ">0</span><span style="color:#808030; ">)</span>  sine_col<span style="color:#808030; ">.</span>back<span style="color:#808030; ">(</span><span style="color:#808030; ">)</span> <span style="color:#808030; ">+</span><span style="color:#808030; ">=</span> <span style="color:#008000; ">10.0</span><span style="color:#800080; ">;</span>  <span style="color:#696969; ">// Inject anomalies</span></span>
<span class="line_wrapper">    <span style="color:#800080; ">}</span></span>
<span class="line_wrapper">    df2<span style="color:#808030; ">.</span>load_column<span style="color:#808030; ">(</span><span style="color:#800000; ">"</span><span style="color:#0000e6; ">sine col</span><span style="color:#800000; ">"</span><span style="color:#808030; ">,</span> <span style="color:#666616; ">std</span><span style="color:#800080; ">::</span><span style="color:#603000; ">move</span><span style="color:#808030; ">(</span>sine_col<span style="color:#808030; ">)</span><span style="color:#808030; ">)</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    and_lof_v<span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">double</span><span style="color:#800080; ">&gt;</span>   anomaly2 <span style="color:#800080; ">{</span> <span style="color:#008c00; ">10</span><span style="color:#808030; ">,</span> <span style="color:#008000; ">1.5</span> <span style="color:#800080; ">}</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper"></span>
<span class="line_wrapper">    <span style="color:#696969; ">// It doesn't work for sine wave data. All the scores are 1.</span></span>
<span class="line_wrapper">    <span style="color:#696969; ">//</span></span>
<span class="line_wrapper">    df2<span style="color:#808030; ">.</span>single_act_visit<span style="color:#800080; ">&lt;</span><span style="color:#800000; font-weight:bold; ">double</span><span style="color:#800080; ">&gt;</span><span style="color:#808030; ">(</span><span style="color:#800000; ">"</span><span style="color:#0000e6; ">sine col</span><span style="color:#800000; ">"</span><span style="color:#808030; ">,</span> anomaly2<span style="color:#808030; ">)</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper">    assert<span style="color:#808030; ">(</span>anomaly2<span style="color:#808030; ">.</span>get_result<span style="color:#808030; ">(</span><span style="color:#808030; ">)</span><span style="color:#808030; ">.</span>empty<span style="color:#808030; ">(</span><span style="color:#808030; ">)</span><span style="color:#808030; ">)</span><span style="color:#800080; ">;</span></span>
<span class="line_wrapper"><span style="color:#800080; ">}</span></span>
<span class="line_wrapper"></span></pre>

  <BR><img src="https://github.com/hosseinmoein/DataFrame/blob/master/docs/LionLookingUp.jpg?raw=true" alt="C++ DataFrame"
       width="200" height="200" style="float:right"/>

</body>
</html>

<!--
Local Variables:
mode:HTML
tab-width:4
c-basic-offset:4
End:
-->
